Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added clickhouse state store #3598

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

middt
Copy link

@middt middt commented Nov 13, 2024

Description

This PR adds a new state store component for ClickHouse, a column-oriented database management system. The ClickHouse state store component provides the following features:

  • Basic CRUD operations for state management
  • Support for TTL (Time-To-Live)
  • ETag support for optimistic concurrency
  • Bulk operations support
  • Username/password authentication
  • Uses ReplacingMergeTree engine for efficient updates

Key implementation details:

  • Uses the official ClickHouse Go driver
  • Implements the full state.Store interface
  • Handles connection management and cleanup
  • Includes comprehensive integration tests
  • Follows Dapr's component development guidelines

Implementation Details

The component includes:

  1. State store implementation (clickhouse.go)

    • Full CRUD operations
    • Bulk operations
    • TTL support
    • ETag support
    • Authentication support
  2. Tests (clickhouse_test.go)

    • Integration tests for CRUD operations
    • Metadata validation tests
    • Authentication tests
    • Cleanup handling
  3. Configuration options:

    apiVersion: dapr.io/v1alpha1
    kind: Component
    metadata:
      name: statestore
      namespace: default
    spec:
      type: state.clickhouse
      version: v1
      metadata:
      - name: clickhouseURL
        value: "tcp://localhost:9000"
      - name: databaseName
        value: "dapr_state"
      - name: tableName
        value: "state_store"
      - name: username
        value: "default"
      - name: password
        value: ""

Checklist

Testing Done

  1. Unit tests for metadata validation
  2. Integration tests for:
    • Basic CRUD operations
    • TTL functionality
    • ETag support
    • Bulk operations
    • Authentication
    • Error handling

Tests were run against ClickHouse v23.8 using the official Go driver.

Additional Notes

  • The component uses ReplacingMergeTree engine for better update performance
  • Includes proper cleanup in tests to avoid test pollution
  • Follows ClickHouse best practices for table design
  • Implements proper connection management and cleanup

@middt middt requested review from a team as code owners November 13, 2024 11:29
Signed-off-by: Mehmet TOSUN <[email protected]>
@middt middt force-pushed the dapr-state-store-clickhouse branch from 3c60d3d to 798489e Compare November 13, 2024 11:35
@middt
Copy link
Author

middt commented Nov 29, 2024

Hello @yaron2
Is there a problem with PR?
We plan to use this in our new application; can you confirm if there is no problem?

@yaron2
Copy link
Member

yaron2 commented Nov 29, 2024

This is a good PR and a great addition to Dapr! I see no issues merging it, however we are currently in the end game time of the 1.15 release and have P0 priorities for this release - so this component will make it into 1.16

@middt
Copy link
Author

middt commented Nov 29, 2024

Thank you for the positive feedback and for considering this PR as a valuable addition to Dapr @yaron2. I understand the current focus on P0 priorities for the 1.15 release. I'm glad to hear that this component will be included in the 1.16 release. If there are any further adjustments or preparations needed to ensure smooth integration in the next release cycle, please let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants